Rigorous Large-Scale Educational RCTs are Often Uninformative : Should We Be Concerned?

There are a growing number of large-scale educational Randomized Controlled Trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the EEF (UK) and the NCEE (US) which evaluated interventions aimed at improving academic achievement in K-12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations (SDs). These sat within relatively large confidence intervals (mean width 0.30 SDs) which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects

Whole School SEND (WSS) Review: study protocol for a two-arm pragmatic parallel cluster randomised controlled trial in 160 English secondary schools

Background The trial will study the effects of the Whole School SEND Review on secondary school pupils in English mainstream education, to understand the impact of the intervention on academic attainment, wellbeing, and school attendance. The Review is designed to facilitate whole-school change through providing enhanced, intensive and sustained support and training in inclusive education for school special educational needs coordinators and leadership teams. The trial will have a specific focus on pupils designated as having special educational needs or disabilities. Methods We recruited 160 English secondary schools (approx. 58,000 pupils across two cohorts) to a two-arm pragmatic parallel cluster randomised controlled trial, with allocation at the school level. Randomisation will be stratified by school region. The primary outcome is attainment in English language (using standardised national test results at 16 years) for pupils designated as having a special educational need (approx. 4000 pupils). Secondary outcomes will be measured for pupils both with and without a special educational need designation and include pupil wellbeing (measured using the Strengths and Difficulties Questionnaire), absences and exclusions, and attainment in Mathematics and English language at 16 years. The intervention will be implemented from July 2021 and analysis of outcomes (for the year 9 cohort) will take place in September 2023, with further analysis (for the year 8 cohort) in September 2024 if the evaluation shows that acceptable implementation fidelity has been achieved. Discussion Pupils with special educational needs represent a significant and often vulnerable part of the secondary school population, are disproportionately likely to be excluded from school, eligible for free school meals, or supported by children’s social care. Despite these multiple important areas of need, school leaders report substantial challenges in making additional provision for this group. Previous research has highlighted the development of inclusive school cultures (rather focusing primarily on targeted individualised approaches) as being important. This trial will investigate how an intervention designed to drive whole school change may lead to outcomes for pupils with and without a special educational needs designation. As such, this trial is expected to make an important contribution to research evidence and to UK educational policy

Self-affirmation reduces the socioeconomic attainment gap in schools in England

Background. Studies in the USA show that school students from some ethnic backgrounds are susceptible to stereotype threat, that this undermines their academic performance, and that a series of virtually zero-cost self-affirmation writing exercises can reduce these adverse effects. In England, however, socioeconomic status (SES) is a much stronger predictor of academic success than is ethnic background. Aims. This study investigates whether self-affirmation writing exercises can help close the SES attainment gap in England by increasing the academic performance of low-SES (but not higher-SES) school students. Sample. Our sample consisted of students aged 11-14 in a secondary school in southern England (N = 562); of these, 128 were eligible for free school meals, a proxy for low SES. Methods. Students completed three short writing exercises throughout one academic year: those randomly assigned to an affirmed condition wrote about values that were important to them, and those assigned to a control condition wrote about a neutral topic. Results. On average, the low-SES students had lower academic performance and reported experiencing more stereotype threat than their higher-SES peers. The selfaffirmation raised the academic performance of the low-SES students by 0.38 standard deviations but did not significantly affect the performance of the higher-SES students, thus reducing the SES performance gap by 62%. The self-affirmation also reduced the level of stress reported by the low-SES school students. Conclusions. The benefits of this virtually zero-cost intervention compare favorably with those of other interventions targeting the SES academic attainment gap

The effectiveness of technology-supported personalised learning in low- and middle-income countries: A meta-analysis

AbstractDigital technology offers the potential to address educational challenges in resource‐poor settings. This meta‐analysis examines the impact of students' use of technology that personalises and adapts to learning level in low‐ and middle‐income countries. Following a systematic search for research between 2007 and 2020, 16 randomised controlled trials were identified in five countries. Studies involved 53,029 learners aged 6–15 years. Coding examined learning domain (mathematics and literacy); personalisation level and delivery; technology use; and intervention duration and intensity. Overall, technology‐supported personalised learning was found to have a statistically significant—if moderate—positive effect size of 0.18 on learning (p = 0.001). Meta‐regression reveals how more personalised approaches which adapt or adjust to learners' level led to significantly greater impact (an effect size of 0.35) than those only linking to learners' interests or providing personalised feedback, support, and/or assessment. Avenues for future research include investigating cost implications, optimum programme length, and teachers' role in making personalised learning with technology effective. Practitioner notesWhat is already known about this topic? Promoting personalised learning is an established aim of educators. Using technology to support personalised learning in low‐ and middle‐income countries (LMICs) could play an important role in ensuring more inclusive and equitable access to education, particularly in the aftermath of COVID‐19. There is currently no rigorous overview of evidence on the effectiveness of using technology to enable personalised learning in LMICs. What this paper adds? The meta‐analysis is the first to evaluate the effectiveness of technology‐supported personalised learning in improving learning outcomes for school‐aged children in LMICs. Technology‐supported personalised learning has a statistically significant, positive effect on learning outcomes. Interventions are similarly effective for mathematics and literacy and whether or not teachers also have an active role in the personalisation. Personalised approaches that adapt or adjust to the learner led to significantly greater impact, although whether these warrant the additional investment likely necessary for implementation at scale needs to be investigated. Personalised technology implementation of moderate duration and intensity had similar positive effects to that of stronger duration and intensity, although further research is needed to confirm this. Implications for practice and/or policy: The inclusion of more adaptive personalisation features in technology‐assisted learning environments can lead to greater learning gains. Personalised technology approaches featuring moderate personalisation may also yield learning rewards. While it is not known whether personalised technology can be scaled in a cost‐effective and contextually appropriate way, there are indications that this is possible. The appropriateness of teachers integrating personalised approaches in their practice should be explored given ‘supplementary’ uses of personalised technology (ie, additional sessions involving technology outside of regular instruction) are common. </jats:sec

Non-clinical community interventions: a systematised review of social prescribing schemes

Abstract Background: This review focused on evaluation of United Kingdom social prescribing schemes published in peer-reviewed journals and reports. Schemes, including arts, books, education, and exercise ‘on prescription’ refer patients to community sources of non-clinical intervention. Method: A systematised review protocol appraised primary research material evaluating social prescribing schemes published 2000-15. Searches were performed in electronic databases using keywords, and articles were screened for evaluation of patient data, referral process, assessment method and outcomes; non-evaluated articles were excluded. Results: Of 86 schemes located including pilots, 40 evaluated primary research material: 17 used quantitative methods including six randomised controlled trials; 16 qualitative methods, and seven mixed methods; 9 exclusively involved arts on prescription. Conclusions: Outcomes included increase in self-esteem and confidence; improvement in mental wellbeing and positive mood; and reduction in anxiety, depression and negative mood. Despite positive findings, the review identifies a number of gaps in the evidence base and makes recommendations for future evaluation and implementation of referral pathways

Improving mathematics in the early years and Key Stage 1: guidance report

'Improving Mathematics in the Early Years and Key Stage 1' reviews the best available evidence to offer five recommendations for developing the maths skills of 3-7-year olds. Recommendations include integrating maths into different activities throughout the day – for example, at registration and snack time – to familiarise children with maths language and make the most of the school day.It also highlights that story and picture books can be a powerful tool for engaging children with basic maths concepts, while board games (such as Snakes and Ladders) are particularly beneficial to developing understanding of numbers.Based on the best available international research – and drawing on the expertise of teachers and academics – this report is packed with examples and case studies to support practitioners.</div